1. Field of the Invention
The invention relates generally to automatic power control and more particularly to remote control methods and devices to reboot computer-based appliances that have frozen, locked-up, crashed, or otherwise become inoperable.
2. Description of the Prior Art
Anthony Coppola describes a computer power management system in U.S. Pat. No. 4,611,289, issued Sep. 9, 1986. A uninterruptable power supply with a limited power storage is connected to supply one or more computers with operating power. A power manager implemented with a microprocessor is connected to signal the computers when power reserves are running short and a graceful shut-down should be executed. This allows data to be saved to disk for use later. The power manager also signals the computers when power conditions have been restored to normal. The computers can signal the power manager to tell it when backup power can safely be cut off.
If such computers were located at some remote site and they shut down, some other means would be necessary to find out why. And if these remote computers were to crash or lock-up due to some software fault, the power manager described by Coppola has no way to be commanded to power cycle the power to any of the computers.
By at least 1991, American Power Conversion (APC) (West Kingston, RI) marketed CALL-UPS, which was a telephone-actuated remote UPS turn-on accessory. The CALL-UPS was intended to work with the APC SMART-UPS to protect computers from brownouts and power outages. Such CALL-UPS connected between a remote computer's modem and the telco subscriber line outlet. When an incoming call was detected by its ring or loop current, the CALL-UPS would command the SMART-UPS to turn on. This, in turn, would cause the computer to boot-up, load application software, and take the call. The power would stay up a few minutes after the call terminated so call-backs could be handled without the reboot delay. Serial data communication only progressed after the computer booted up, loaded the application software, and finished the modem handshaking. The so-called CALL-UPS-IL was introduced about February of 1994 and it enabled a locked-up LAN service to be remotely corrected by rebooting crashed devices through an out-of-band modem link.
A very similar but much earlier arrangement is described by Guido Badagnani, et al., in U.S. Pat. No. 4,051,326, issued Sep. 27, 1977. A call ring signal is used to turn on a data terminal. Once the data terminal completes its initialization, it sends a ready-to-receive signal and a data conversation can begin. Another telephone-activated power controller is described by Vincent Busam, et al., in U.S. Pat. No. 4,647,721, issued Mar. 3, 1987.
Another one like these is described by Arthur P. Ferlan, in U.S. Pat. No. 4,206,444, issued Jun. 3, 1980, and titled REMOTE POWER CONTROLLER UTILIZING COMMUNICATION LINES. The stated objective is to allow remote computers to turn off and be powered up only when needed. For example, when another computer calls in and wants service. But here encoded messages are used on dedicated telephone lines, e.g., Dataphone Service. The remote verifies who is calling, and allows access only if authorized. If authorized, the remote computer is powered up.
A kind of alarm clock was added to this basic configuration by Raymond A. Oliva, et al., their device for controlling the application of power to a computer is described in U.S. Pat. No. 4,701,946, issued Oct. 20, 1987. The alarm clock can turn the remote computer on and off according to a preset schedule.
Two of the present inventors, Carrel Ewing and Andrew Cleveland, described technology along these general lines in PCT International Publication Number WO 93/10615, published May 27, 1993. This is a system for protecting and restarting computers and peripherals at remote sites which are accessible by telephone communication. They also filed U.S. patent application Ser. No. 08/061,197, on May 13, 1993, and now abandoned, for a REMOTE POWER CONTROL SYSTEM FOR COMPUTER AND PERIPHERAL EQUIPMENT. Such specifically described power-cycling to reset a remote computer that had become hung up.
Things have changed quite a lot since then. Computer-based appliances are now required to be on all the time. Any down-time is costly. But computers being what they are, they lock up occasionally and a power-on reset is about the only way to generate a reboot. When such computer-based appliances are network servers, routers, and bridges located at telco modem-farm locations, it isn't practical to send a technician to the site to force the operating power on-off-on. Much more than a simple phone call to a dial-up number is needed too, an accidental reboot could cause serious damage to user's data and the service provider's goodwill.
Enterprise networks exist to support large world-wide organizations and depend on a combination of technologies, e.g., data communications, inter-networking equipment (frame relay controllers, asynchronous transfer mode (ATM) switches, routers, integrated services digital network (ISDN) controllers, application servers), and network management application software. Such enterprise networks can be used to support a large company's branch offices throughout the world, and, as such, these networks have become mission critical to the functioning of such organizations. Masses of information are routinely expected to be exchanged, and such information exchanges are necessary to carry on the daily business of modern organizations. For example, some international banks have thousands of branch offices placed throughout Europe, Asia and the United States that each critically depend on their ability to communicate banking transactions quickly and efficiently with one another and headquarters.
A typical enterprise network uses building blocks of router and frame relay network appliances mounted in equipment racks. Such equipment racks are distributed to remote point of presence (POP) locations in the particular network. Each equipment rack can include frame relay controllers, routers, ISDN controllers, servers and modems, etc., each of which are connected to one or more power sources. The value of POP equipment can range from $200,000 to $500,000, and the number of individual devices can exceed a thousand.
Many enterprises rely on an uninterruptable power supply (UPS) to keep their network appliances operational. Many network appliances are typically connected to a single UPS, and this sets up a problem. When an individual router locks up, the router's power cannot be individually cycled on and off externally at the UPS because it is connected to a multiple power outlet. The recovery action choices available to the network control center operator thus do not include being able to reinitialize the individual equipment through a power interruption reset. The network operator could command the UPS to power cycle, but that would reset all the other attached devices that were ostensibly operating normally and carrying other network traffic. Another option is to dispatch someone to the remote location to reset the locked-up device. Neither choice is an attractive solution.
In large organizations that have come to depend heavily on enterprise networks, there is great pressure to develop ways to control costs and thus to improve profits. Organizational down-sizing has been used throughout the corporate world to reduce non-network costs, and that usually translates to fewer technical people available in the right places to support large and complex in-house global networks. Such reduced repair staffs now rely on a combination of centralized network management tools and third-party maintenance organizations to service their remote POP sites. The costs associated with dispatching third-party maintenance technicians is very high, and the dispatch and travel delay times can humble the business operations over a wide area for what seems an eternity.
Global communication network operators, located at a few centralized network management centers, are relying more and more on automated network management applications to analyze, process, display and support their networks. An increasing number of network management software applications are being marketed that use open-system standardized protocols. Particular network application tool software is possible to report lists of the network appliances, by location, and can issue trouble lists and keep track of software versions and releases. Simple network management protocol (SNMP) applications are conventionally used to issue alarms to central management consoles when remote network appliances fail.
SNMP is conventionally used to send messages between management client nodes and agent nodes. Management information blocks (MIBs) are used for statistic counters, port status, and other information about routers and other network devices. GET and SET commands are issued from management consoles and operate on particular MIB variables for the equipment nodes. Such commands allow network management functions to be carried out between client equipment nodes and management agent nodes. The agent nodes can issue alert or TRAP messages to the management center to report special events.
SNMP is an application protocol for network management services in the internet protocol suite. SNMP has been adopted by numerous network equipment vendors as their main or secondary management interface. SNMP defines a client/server relationship, wherein the client program, a “network manager”, makes virtual connections to a server program, an “SNMP agent”, on a remote network device. The data base controlled by the SNMP agent is the SNMP management information base, and is a standard set of statistical and control values. SNMP and private MIBs allow the extension of standard values with values specific to a particular agent. Directives issued by the network manager client to an SNMP agent comprise SNMP variable identifiers, e.g., MIB object identifiers or MIB variables, and instructions to either GET the value for the identifier, or SET the identifier to a new value. Thus private MIB variables allow SNMP agents to be customized for specific devices, e.g., network bridges, gateways, and routers. The definitions of MIB variables being supported by particular agents are located in descriptor files, typically written in abstract syntax notation (ASN.1) format. The definitions are available to network management client programs.
SNMP-based network management systems (NMS) can be implemented with Compaq INSIGHT MANAGER, Novell NETWARE, Hewlett-Packard OPENVIEW, Castlerock SNMPC, Banyan VINES, Artisoft LANTASTIC, Microsoft WINDOWS, SunNet MANAGER, IBM AS/400, etc. Specific control of an agent is traditionally afforded by hardware manufacturers by supplying MIB extensions to the standardized SNMP MIB library by way of source-text files on floppy disks or compact disks (CD's). These MIB extensions load on the NMS, and an assigned IP-address for the agent is entered-in by a user at the NMS. Connecting the agent and the NMS to a properly configured network is usually enough to establish communications and control.
In 1994, American Power Conversion (West Kingston, RI) marketed a combination of their SMART-UPS, POWERNET SNMP ADAPTER, MEASURE-UPS, and an SNMP-based management station. POWERNET SNMP agents were used to generate traps or alarms for attention by the management station. The SNMP agents were described as being able to supply real-time UPS status and power-quality information, e.g., UPS run-time, utility-line voltage, and UPS current load.
In 1996, American Power Conversion was marketing their MASTERSWITCH embodiment that comprises a single rack-mountable box with eight relay-controlled power outlets on the back apron. A built-in 10 Base-T networking plug allows connection to a LAN. It further includes an embedded SNMP agent responsive to the networking plug that can control individual power outlets. A Telnet agent was also included. Revisions of the MASTERSWITCH that appeared by 2000 further included a hypertext transfer protocol (HTTP) agent that can generate information and control webpages on a logged-in web browser. SNMP traps were relied on to generate unsolicited alarm inputs. Automatic IP-address assignment is provided by a Bootup process.
By at least 1998, American Power Conversion began marketing a “complete enterprise power management system”. A POWERNET manager controls SMART-UPS devices over a network using SNMP. An SNMP agent is associated with each controlled SMART-UPS and a graphical user interface (GUI) on the manager allows a user to see the power status of each SMART-UPS. Shutdowns and reboots of individual SMART-UPS sites are initiated from the GUI. The POWERNET EVENT ADAPTER converts SNMP traps into events that are reported in a GUI, e.g., the TIVOLI ENTERPRISE CONSOLE (TEC). In 1998, voltage, current, temperature, and relative humidity were being reported, e.g., by MEASURE-UPS, and displayed in the POWERNET MANAGER GUI.
All such patents and patent applications mentioned herein are incorporated by reference.